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This paper discusses a methodology and modeling capability that probabilistically 
evaluates the likelihood and impacts of delays in advanced technology development prior to 
the start of design, development, test, and evaluation (DDT&E) of complex space systems. 
The challenges of understanding and modeling advanced technology development 
considerations are first outlined, followed by a discussion of the problem in the context of 
lunar surface architecture analysis. The current and planned methodologies to address the 
problem are then presented along with sample analyses and results. The methodology 
discussed herein provides decision-makers a thorough understanding of the schedule 
impacts resulting from the inclusion of various enabling advanced technology assumptions 
within system design. 
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I. Introduction 

T hroughout historical design and development of complex systems, unforeseen delays have frequently occurred 
due to engineering challenges, funding shortfalls, and inherent design optimism that have led to severe impacts 
on projected development schedules and costs. This behavior is damaging to all development investments but is 
particularly threatening to the development of multi-system architectures or infrastructures. Schedule relationships 
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are pervasive when various systems within such architectures are dependent on each other for either inclusion (i.e., 
subsystem within larger system) or for production requirements (i.e., concurrent production plans). Schedule delays 
within one system of a multi-system architecture will at best cause holding delays to other related systems, while at 
worst could lead to reduced and inadequate funding for other systems due to reallocation of resources to mitigate the 
cause of the original delay. In addition to delays inherent with traditional system development, pre-requisite 
development of advanced technologies is severely prone to experiencing delays due to additional work aimed at 
squeezing every bit of promised benefits from such investment, which often is crucial for the system’s overall 
feasibility. By their very nature, advanced technologies carry a large amount of uncertainty associated with their 
benefits (e.g., performance improvements) as well as their costs and schedules. Historical examples have often 
demonstrated that entire development projects can be delayed while issues are worked out with assumed technology 
development paths. 


II. Technology Development Challenges 

A. Advanced Technology Development Considerations within Design 

The inclusion of advanced technologies within complex system design presents design engineers with numerous 
challenges due to the uncertain nature of how such technologies will impact their systems. Not only do the positive 
impacts have to be predicted (in the form of improved performance characteristics, lessened maintainability 
demands, etc.), but the associated development cost and development risk have to be appropriately considered. It is 
easy to focus on the benefits since doing so typically expands the design space and in many cases actually enables 
design feasibility. However the benefits in many cases can quickly be eroded by increased costs, schedule, and 
associated uncertainties, requiring careful cost/benefit analyses. In addition, the benefits and penalties associated 
with advanced technologies are very often highly interrelated. When a technologist advertises the benefits of a 
particular advanced technology, an assumption is made that such return on the investment (the further development 
of said technology being the investment) is based on some certain threshold of funding. If such a development 
program is funded at a lesser rate or schedule, the claimed benefits of inclusion can fall critically short. This, in 
addition to the inherent nature of technology benefits to fall short due to elusive technological breakthroughs, 
spurious starting assumptions, and other considerations, fosters a scenario where costs can far exceed promised 
benefits. 

This is not to say that the inclusion of advanced technologies within complex system design is to be avoided. In 
fact the exact opposite is often warranted for it allows genuine progress to be made within scientific and engineering 
communities and enables systems to achieve increasingly demanding performance levels. Systems with razor thin 
mass margins such as space transportation and lunar surface systems almost always require advanced technology 
fruition to enable feasibility. Design decisions made early during the design process tend to drive overall system 
behavior, so systems engineering is not complete without appropriate focus on technology development. Since 
technology development is such a prominent life cycle consideration in its own right, it can be considered a separate 
discipline from traditional systems engineering. “Technology Engineering” is the subject of both a significant 
amount of academic research and dedicated development programs (dedicated in the sense that they are funded and 
tracked separately from the system(s) that they will ultimately be incorporated in). 1 There is also a wide range of 
consideration given to how to “select” which technologies will be incorporated into a system’s development; thus 
there is a myriad of published technology assessment and prioritization techniques. 2 * " 7 

The assessment of advanced technology inclusion has been discussed from numerous perspectives, but Stanley 
and Wilhite lay out a technology engineering framework that accurately portrays the various phases of identifying 
and including space transportation technologies within the larger realm of systems engineering: 1 

1. Technology Requirements Development and Analysis 

2. Technology Prioritization and Initial Selection 

3. Technology Program Planning 

4. Technology Program Monitoring and Management 

5. Technology Integration and Insertion 

“Technology Requirements Development and Analysis” deals with the identification of system requirements gaps in 
combination with potential technology development solutions that could mitigate such gaps. Once the gaps have 
been identified and a pool of potential technology solutions has been identified, the “Technology Prioritization and 
Initial Selection” phase contains the actual prioritization and screening of candidates leading to a technology 
investment portfolio. The “Technology Program Planning” and “Technology Program Monitoring and 
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Management” phases capture the programmatic considerations of laying out and monitoring appropriate roadmaps 
and milestones associated with the aforementioned portfolio. The final “Technology Integration and Insertion” 
phase pertains to the final verification that the selected technology development paths have matured to the 
appropriate level and will provide promised benefits while maintaining planned costs, and that they are ready for 
inclusion within larger system DDT&E. 

Throughout all five phases a common theme exists in that all technology candidates have to be measured using 
quantifiable impact characterizations that allow comparison to target system requirements. Stanley refers to the use 
of Technology Technical Performance Measures (TTPMs) that represent a “subsystem- or component-level 
parameter that is readily measurable during the implementation of a given technology project by a knowledgeable 
technologist”, while other research uses other names such as “k-factors” to represent the same concept. 4 These 
impact characterizations quantitatively relate how a given technology will impact a particular system in terms of 
performance, reliability, maintainability, development cost, operational costs, and so forth. They also allow 
engineers to understand the trade-offs associated with the inclusion of such technology, thus enabling the 
prioritization process. Since so much uncertainty surrounds the fruition of advanced technologies, all such impact 
characterizations should take the form of distributions as opposed to point prediction estimates. In addition to the 
impacts to a larger system, each technology development candidate carries additional characteristics that are highly 
related to, but somewhat independent of, the system impacts. These characteristics are related more to a 
development program itself required to mature said technology to the point where it can be incorporated within a 
larger system. These characteristics include, but are not limited to, the current maturity level of the technology, the 
cost and schedule required to mature it to the point where it can be used, and so forth. This latter set of 
characteristics, pertaining to the development effort itself, is of interest in understanding the potential schedule 
impacts such development efforts could have on full DDT&E schedules for complex systems. 

B. Relationship Between Technology Development and Traditional System Development 

To understand how various advanced technologies could potentially influence a system’s development, a 
characterization of the current maturity level of each technology is used. The Technology Readiness Level (TRL) 8 
provides standardized insight into the maturity level and is commonly used throughout the aerospace community. 
Mathias et al. propose an alternative scale that is more consistent with how historical technology development data 
can be collected; i.e., is more in-line with development program milestones. 9 The two scales are similar in nature 
and can be seen in Fig. 1. 
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Figure 1. Translation Between TRL and TR Bins. 


3 

American Institute of Aeronautics and Astronautics 





Both the traditional TRL scale and the TR Bin scale capture the full life cycle of a technology’s development 
through actual production and operation of the system or vehicle that it ultimately supports. However, full vehicle 
DDT&E generally assumes that all advanced technologies are at a TRL level of at least 6 (i.e., system/subsystem 
model or prototype has been demonstrated in a relevant environment) so the upper portion of the traditional TRL 
scale (TRLs 7-9) can be assumed to be contained within the vehicle’s DDT&E effort. This can also be seen in the 
bottom right portion of Fig. 1. where the red TRL 6 and TRL 8 identifiers identify how they relate to vehicle 
DDT&E. The TR Bin identifiers are also included so it can be deduced that a TRL value of 6 equates to a TR Bin 
value of 4, and that both indicate how mature a technology should be in order to be included in vehicle DDT&E. 
Likewise, TRL 8 equates to TR Bin 5, and indicates the assumed maturation of such technology once the vehicle is 
fully developed and ready for production. It is crucial to understand these relationships when attempting to forecast 
expected behavior of technology development programs and their relationship to vehicle DDT&E efforts. 

C. Using Historical Data to Forecast Future Behavior 

Since uncertainty is pervasive throughout all advanced technology considerations, appropriate modeling requires 
distributions around virtually every assumption. The quantification of actual benefits and costs associated with 
inclusion of technologies within a complex system or vehicle requires stochastic analysis in addition to the 
understanding of how each technology will impact the system from a technical perspective. A technical 
understanding of the system and how it is impacted by technology inclusion is typically based on traditional systems 
engineering disciplines. However, the characteristics associated with the technology development efforts 
themselves, such as the cost and schedule to mature a technology to TRL=6/TR Bin = 4, cannot be solely based on a 
technical understanding of the system or technology. Each historical example “represents a ‘real’ scenario that 
included the effects of real budgets, political environments, and technical difficulties”. 9 By relating a future 
advanced technology candidate to similar past efforts, an understanding of expected behavior can be deduced. The 
additional challenge therein exists in that it is extremely difficult to collect historical data in such a way to 
completely understand what drove each example in terms of cost and schedule. Several historical data 
considerations can be seen in Table 1. 


Table 1. Considerations in Understanding Historical Technology Development Examples. 


Historical Data Consideration 

Description 

Adequacy of Funding 

Was the technology development funded at an optimal level? 

Push vs. Pull 

Was it developed as a need or for its own benefit? 

Enabling vs. Enhancing 

Did it enable an actual system’s feasibility, or did it just 
enhance some technical attribute? 


The adequacy of funding is important for it provides insight into whether a technology development effort was 
funded at an optimal level versus either a crash-effort or a case where the effort was kept alive at some minimal 
level until additional money became available. Another consideration is whether a technology development effort 
was required by a particular vehicle being developed versus being developed in hopes of finding a future 
application. Finally, if a technology was included on a vehicle after maturing to the appropriate point, it should be 
noted whether or not it was an enabling technology for vehicle feasibility versus an enhancing technology included 
to boost some attribute, such as a performance metric that already met design requirements. These considerations are 
examples of issues that may have significantly driven past technology development behavior. Of course it is difficult 
to assess actual causality, but historical data collection should attempt to capture these and similar characterizations. 

III. Problem Context 

In January of 2004 the President of the United States announced a new space exploration vision for NASA to 
extend human presence across the solar system, starting with a human return to the Moon by the year 2020, in 
preparation for human exploration of Mars and other destinations. To implement this vision, the development of 
innovative technologies, knowledge, and infrastructures are needed both to explore and to support decisions about 
the destinations for human exploration. 

The Constellation Systems Program was launched to implement the findings and recommendations assessed by 
the Exploration Systems Architecture Study (ESAS) team into a sustainable human space program. 10 The success of 
the program is critically dependent on the successful development of various new technologies both originally 
envisioned during ESAS and later revised as requirements changed and designs matured. Success is also dependent 
on the successful development and integration of a multi-element architecture since the original exploration 
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architecture consisted of a human-rated Crew Exploration Vehicle (CEV), a Lunar Surface Access Module (LSAM), 
a human-rated Crew Launch Vehicle (CLV), a Cargo Launch Vehicle (CaLV) which includes the Earth Departure 
Stage (EDS), Ground Systems, Missions Systems, and future destination surface systems such as habitats, power 
systems, rovers, science equipment, robotic systems, and resource utilization systems that enable the crewmembers 
to live, work and explore the surface of other worlds. Many of these elements have since become project offices 
within the Constellation Program, and have been given new nomenclatures (e.g., the Orion Crew Vehicle (CEV), 
Ares Launch Vehicles (CLV and CaLV), and the Altair Lunar Lander). 

The multi-element aspect of the Constellation Architecture lends itself to numerous instances where a delay in 
one element’s development could lead to significant delays in other elements. The upcoming Orion and Ares I 
combination is an example since a delay in either could potentially delay the Initial Operating Capability (IOC) date 
for the other. Another pervasive example of this interrelationship is within the lunar surface sub-architecture, which 
is part of the Constellation Program, but can be thought of as an architecture in its own right due to various systems 
that have to be both developed and delivered to the lunar surface in coordinated fashion. Of course all lunar systems 
are dependent on the development of the transportation system to deliver them in the first place, but assuming the 
transportation capability is available as planned, a large amount of consideration has to be given to how delays 
within certain lunar systems could impact others. 

When planning a lunar architecture, decision makers have to balance an appropriate amount of advanced 
technology inclusion with potential inter-related delays that would be expected. A distribution of the time required 
to reach set milestones within system development projects is needed to account for schedule uncertainty associated 
with technology maturation. These distributions would provide insight into how the inclusion of various advanced 
technologies may impact planned costs and schedules. In addition to milestone distributions, supporting data could 
include statistical characteristics such as confidence intervals that indicate the credibility of an estimate, and 
sensitivity analysis results which could verify the stability of the assumptions. 

To arrive at this level of understanding, one has to determine the driving factors by probabilistically combining 
the likelihood and impacts of delays in advanced technology development prior to the start of DDT&E for various 
architecture elements. The intent would be to understand differences between planned and estimated initial operating 
dates of systems within an architecture. This could be accomplished by comparing historical examples of technology 
maturation efforts to identify key driving factors which can then be used to make predictions on how future 
technology development investments would pan out. 

Both planned and historical technology data are required since the analysis would compare technology 
investments (or investment candidates) currently being planned versus how similar examples have behaved in the 
past. In the case of the NASA’s Constellation lunar surface architecture, a list of identified technologies associated 
with the various architecture elements would be analyzed along with inter-relationships between the elements 
themselves and the technology investment paths. Within the planned list of technology investments, data can be 
assembled from several technology sectors, such as: propulsion, suit development, power generation, radiation 
shielding, cryogenic fuel storage, habitat development, and surface mobility. However, most new technologies 
cannot be directly matched to any past projects. Hence, surrogate past technology development projects could be 
used within the analysis as indicators of how the current efforts may behave. For example, consider the construction 
of a lunar habitat. Thus far, there has never been a permanent habitable structure assembled on the moon. On the 
other hand, several extreme environment research habitats exist on earth, such as arctic and undersea habitats that 
are currently being occupied and used by scientists, researchers, and explorers. The technologies utilized and the 
lessons learned from technology inclusion within these systems could be applied to the development of a lunar 
habitat. 

The success of the space exploration program is dependent on managing the development of new technologies 
and accounting for the uncertainty inherent within the process. Historical design and development of complex 
systems prove that there is no assurance that by just identifying and funding a technology, it will progress on time to 
have the necessary performance capabilities and stay within the allocated budget. The following discussion lays out 
an emerging capability to leverage historical technology development efforts to understand future investments. 

IV. Methodology 

The basic methodology for quantitatively estimating the schedule risk of a project element as a function of TRLs 
was described by Mathias et al. Q The current study extends that approach to model large projects encompassing 
multiple elements where each element may require a number of pre-requisite Advanced Technology Development 
Projects (ATDPs). The five calculation steps of this analysis are: 
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1. Create a project level work breakdown structure (WBS) with its constituent elements (e.g.„ 
Unpressurized Rover) 

2. Identify the pre-requisite ATDPs for each element in the WBS 

3. Quantify the following characteristics of each ATDP: 

a. Planned start date of the ATDP 

b. TRL level at the start date of the ATDP 

c. Time needed to mature from one TRL level to another 

4. For each element, estimate the date when all of its required ATDPs reach the point where they are ready 
to begin DDT&E. On the Mankins TRL scale, this corresponds to TRL=6 which is defined as 
“System/subsystem model or prototype demonstration in a relevant environment (ground or space)”. If 
this date is later than the date originally planned, the difference between the planned and actual dates is 
the schedule slip for that element. 

5. Assess the delays associated with the individual elements to estimate the delay of the overall project 

The first step is to assemble a WBS of the overall project. The highest level of the WBS typically corresponds to 
the major elements or functional requirements of the project. The WBS may be identical or similar to the WBS used 
by project management and/or cost analysts. 

After the WBS is defined, the pre-requisite ATDPs for each element are identified and characterized with the 
assumption that each identified ATDP will achieve a TRL of 6 by the time the element’s DDT&E begins. The list of 
ATDPs may be compiled using input from project engineering and staff familiar with the ATDPs, or could be the 
result of a technology prioritization process as mentioned in Section 11 of this paper. Some ATDPs may be pre- 
requisites to more than one project element. This step may identify gaps in technology where additional ATDPs 
should be initiated. 

The third step is to characterize each ATDP with three pieces of information: 

1. Planned start date of the ATDP 

2. TRL of the ATDP on its start date 

3. Time needed for the ATDP to mature from its initial TRL (= j) to its required TRL (=6). The time may 
be estimated by summing the times needed to transition from one TRL to the next, or 

T(j — » 6) = T(j -A./ + l) + ... + 7X5 -A 6) (1) 

When an element requires one or more ATDPs, the start date of DDT&E for that element is equal to the latest 
(maximum) date that any individual ATDP achieves the desired TRL=6. If the element consists of n ATDPs, the 
time to reach TRL= 6 is given by 

T(6, Element) = max(T (6, A TDP X ), T(6, A TDP 2 ),..., T(6, A TDP n )) (2) 

Figure 2 uses two elements named E t and E 2 to illustrate steps 1, 2 and 3. Element Ej requires three ATDPs, 
named A 1; A 2 and A 3 . E 2 only requires A 3 which was also required by Ej. The TRL values of the ATDPs are shown 
in the circles on the timelines for the ATDPs. Ej starts its DDT&E when A 2 reaches TRL=6 because A 2 is the ATDP 
that finishes last. Similarly, E 2 starts its DDT&E when A 3 reaches TRL=6. 
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Figure 2. Timeline of WBS Elements and Pre-requisite ATDPs. 

Since ATDPs are generally first of a kind engineering designs, historical data on development times of identical 
items obviously do not exist. The best sources of information are historical data from analogous projects or 
subjective judgments by subject matter experts. Data on analogous projects may be pulled from diverse sources such 
as official project documentation, scientific journals, newspaper items, and magazine articles. These sources 
typically do not provide neatly defined TRL values at key milestones. Instead, they may be vague, inconsistent and 
subject to interpretation by the analysts. Even when there are a number of independent and consistent sources, gaps 
may still exist in the project chronology. 

Once a set of data has been compiled for an advanced technology, regression analysis is used to reduce that data 
into a set of descriptive statistics such as the mean, variance, and percentiles of the time needed to transition from 
one TRL level to another. 

The data required for steps 1, 2 and 3 may be organized as shown in Table 2, Table 3, and Table 4. Table 2 
shows which ATDPs are assigned to each element as well as the initial TRL and start date of the ATDPs. Table 3 
shows which historical data sets are used by each ATDP in calculating the time to reach TRL=6. Table 2 and Table 
3 are linked through the ATDPs which are listed in the first column of each table. Table 4 represents the historical 
data. Each row is a historical data set and each cell is the time needed to jump from one TRL to another. The time 
may be a single value or a probability distribution of likely times. 
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Table 3. ATDP to Historical Data Set Matrix. 
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Table 4. TRL Transition Times. 


Data 

Sets 

Time to Transition from One TRL Level to the Next 

TRL (1 to 2) 

TRL (5 to 6) 

TRL (8 to 9) 

hi 

t(hi, 1 ->2) 

t(h i, 5 ->6) 

t(hj, 8 ->9) 

^2 

t(h 2 , 1 ~>2) 

t(h 2 , 5 ->6) 

t(h 2 , 8~>9) 


h L 

t(h L , 1 ~>2) 

t(h L , 5 ->6) 

t(h L , 8 ~>9) 


In a deterministic analysis, the average time needed to reach TRL=6 is estimated as the sum of the average times 
for each of the individual TRL transition steps. If the estimated date when the element reaches TRL=6 exceeds the 
originally planned date, the start of DDT&E is delayed. The schedule slip is the difference between the estimated 
and originally planned dates, or 

Delay =0 if T (6, ATDP ) < T (6, planned ) 

= T (6, estimated) -T (6, planned) if T (6, ATDP) > T (6, planned) ^ 

For probabilistic calculations, a numerical simulation such as Monte Carlo is normally required. In each trial of 
the simulation, the time it takes the ATDPs to reach TRL=6 is obtained by sampling the times needed to transition 
from one level to the next and summing the results. Probabilistic results are presented either as a probability 
distribution function (PDF) or cumulative distribution function (CDF). 

In the simplest case of no time dependencies between the elements, the schedule slip of the overall project is 
determined by adding the delays and penalties of the various elements. For a probabilistic analysis, the new schedule 
is obtained by simulating a large number of scenarios using the probability distributions for the ATDP maturity 
times. 


V. Sample Analysis Results 

This section describes how the methodology outlined was used to analyze schedule risk in NASA’s Constellation 
project. Past research by Mathias et al. Q and Forgie and Evans 11 has included collection of historical data with 
regression analysis to organize it into various bins applicable to current Constellation needs. The data in this sample 
are notional and are not representative of the actual project values. The numerical algorithm, including random seed 
generation, was implemented in Microsoft Excel® . 

The sample analysis evaluates the schedule risk of the planned IOC date for the Constellation program’s lunar 
architecture if pre-requisite ATDPs are delayed. To simplify this analysis, the following assumptions have been 
made: 

• The time needed to transition from one TRL to another within a particular ATDP is normally 
distributed 

• TRL transitions from DDT&E to IOC are not modeled 
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The WBS for this sample analysis contains the following notional surface system elements: Habitat, Pressurized 
Logistics Module, Unpressurized Rover, Pressurized Rover, and Power System. 

The data used in this sample analysis are shown in Tables 5-7. Table 5 shows the five elements of the WBS 
along with the planned duration of the DDT&E and planned IOC date. In this example, the elements assume a 
common IOC date of 2020 and a standard DDT&E duration of 4 years. The planned DDT&E start date of 2016 is 
obtained by subtracting the DDT&E duration from the IOC date. All of the ATDPs associated with an element must 
reach TRL=6 (or greater) before the planned DDT&E start date. 

Table 6 lists the ATDP elements and the WBS elements. ATDPs are linked to a WBS element by entering a non- 
zero value into the cell where the WBS element column and ATDP row intersect. For example, the pre-requisite 
ATDPs for the Habitat element are Al, A2, A3, A4 and A5. This table also shows the following information for 
each ATDP: 


• historical data set which characterizes the time to mature from one level to another 

• the TRL of the ATDP at the start of the ATDP 

• the start for the ATDP 

Table 7 is the database of notional historical transition times used in this analysis. This database uses the TR 
“bins” structure originally proposed by Mathias. There are 6 bins instead of 9 levels as in the Mankins formulation. 
The conversion between TR bins and TRLs was shown in Fig 3. Since most of the literature uses the 9 level TRL 
formulation, this model assumes that the initial technology readiness of an ATDP is also expressed on the 9-level 
scale. The conversion to the 6-bins is performed internally by the software and is completely transparent to the user. 
It can also be seen that the list of historical technologies in Table 7 is not referenced in its entirety in Table 6. This 
demonstrates that the ATDP list only pulls historical comparisons that are germane, and that the entire set of 
historical data is not be needed. 

Table 5 also shows the likelihoods that the elements have reached IOC as a function of time with delays 
attributed to the multiple advanced technology developments associated with each element. In the example, there is 
a 68% likelihood that the Habitat element achieves IOC by the end of 2020 as originally planned. The likelihood 
increases to 88% and 95% by 2021 and 2022 respectively. The pressurized logistics module has only a 38% 
likelihood of being at IOC by its originally planned date of 2020. The likelihood rises gradually to 60%, 77% and 
89% in 2021, 2022 and 2023, respectively. Note that likelihood for reaching IOC prior to the planned date is zero 
because the current model does not accelerate the DDT&E start date if the pre-requisite ATDPs are completed 
earlier than planned. 


Table 5. Example Results per Architecture Element (Notional). 



Likelihood of Reaching IOC (=completion of DDT&E) 

Constellation Elements 

DDT&E 

(yrs) 

IOC 

2018 

2019 

2020 

2021 

2022 

2023 

2024 

2025 

2026 

2027 

Habitat 

4.0 

2020 

0.00 

0.00 

0.68 

0.88 

0.95 

0.98 

0.99 

1.00 

1.00 

1.00 

Pressurized Loqistics Module 

4.0 

2020 

0.00 

0.00 

0.38 

0.60 

0.77 

0.89 

0.95 

0.98 

0.99 

1.00 

Unpressurized Rover 

4.0 

2020 

0.00 

0.00 

0.52 

0.69 

0.82 

0.90 

0.95 

0.98 

0.99 

1.00 

Pressurized Rover 

4.0 

2020 

0.00 

0.00 

0.47 

0.63 

0.77 

0.87 

0.93 

0.97 

0.98 

0.99 

Power System 

4.0 

2020 

0.00 

0.00 

0.43 

0.66 

0.82 

0.92 

0.97 

0.99 

1.00 

1.00 
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Table 6. ATDP to Element Relationships (Notional). 

Yes Yes Yes Yes Yes 

1 
1 

1 I 
1 

1 I 1 

1 
1 
1 

1 
1 
1 
1 

1 
1 
1 

1 
1 

p i 
1 
1 


Advanced 
Tech Project 

Category 

Data 

Set 

A1 

Habitat 

H11 

A2 

Habitat 

H12 

A3 

Habitat 

H13 

A4 

Habitat 

H14 

A5 

Habitat 

H15 

A6 

Pressurized Logistics Module 

H16 

A7 

Pressurized Logistics Module 

H17 

A8 

Pressurized Logistics Module 

H18 

A9 

Unpressurized Rover 

H20 

A10 

Unpressurized Rover 

H21 

All 

Unpressurized Rover 

H22 

A12 

Unpressurized Rover 

H23 

A13 

Pressurized Rover 

H24 

A14 

Pressurized Rover 

H25 

A15 

Pressurized Rover 

H26 

A16 

Power Systems 

H31 

A17 

Power Systems 

H32 

A18 

Power Systems 

H33 

A19 

Power Systems 

H34 

A20 

Power Systems 

H35 


Table 7. Historical TR Bin Transition Times (Notional). 




Bin2 -> Bin3 

Bin3 - 

> Bin4 

Category 

Data 

Set 

Mean 

Std Dev 

Mean 

Std Dev 

Habitat 

H11 

0.348 

0.181 

1.649 

0.868 

Habitat 

H12 

0.95 

0.37 

2 00 

0.37 

Habitat 

H13 

1.33 

0.39 

1.20 

0.76 

Habitat 

H14 

0.67 

0 62 

1.18 

0.37 

Habitat 

H15 

1.14 

1.43 

0.70 

0.65 

Pressurized Logistics Module 

H16 

0.933 

0 01 

0.941 

1.428 

Pressurized Logistics Module 

H17 

1.92 

0.92 

0.56 

0.77 

Pressurized Logistics Module 

H18 

1.88 

1.00 

0 91 

0.00 

Pressurized Logistics Module 

H19 

0.24 

0.76 

0.44 

0.46 

Unpressurized Rover 

H20 

0.726 

0.705 

0 357 

1.448 

Unpressurized Rover 

H21 

0.31 

0.67 

1.31 

1.39 

Unpressurized Rover 

H22 

0.69 

0.39 

0.78 

0.61 

Unpressurized Rover 

H23 

0.72 

1.16 

0.32 

0.02 

Pressurized Rover 

H24 

1.363 

0.838 

0 447 

1.363 

Pressurized Rover 

H25 

0.73 

1.30 

0.47 

0.98 

Pressurized Rover 

H26 

0.99 

0.70 

1 97 

1.13 

Pressurized Rover 

H27 

1 64 

1.09 

1 06 

1.34 

Pressurized Rover 

H28 

1.09 

0.47 

1.25 

0.68 

Pressurized Rover 

H29 

1.58 

0.68 

0.20 

1.20 

Power Systems 

H30 

0 322 

0 013 

0471 

1.424 

Power Systems 

H31 

0 01 

0.19 

0.17 

0.66 

Power Systems 

H32 

0.92 

0.54 

1.39 

1.22 

Power Systems 

H33 

1.50 

0.33 

1.99 

0.33 

Power Systems 

H34 

1.16 

0.55 

1 94 

0.98 

Power Systems 

H35 

1.94 

0.52 

0.86 

1.12 


The likelihood of achieving the planned IOC dates as shown in Table 5 is plotted as the S-curves in Fig. 3. This 
plot allows the analysts to quickly assess the likelihood of achieving IOC on any date or to estimate the date when 
an element will reach a given likelihood. The inset in Fig. 4, which is generated by analyzing resulting data from the 
probabilistic runs, shows how each historical ATDP of the pressurized logistics module contributes to the delay of 
that element. These data may be used to prioritize efforts to meet the planned schedule. In this example, priority 
should be given to the ATDP named A7 because it has a 65% chance of causing a delay. If A7 is mitigated to the 
point that it only contributes 40% or less of the delay, work should also be undertaken to improve A8 because it 
currently contributes 40% of the delay. 
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Figure 3. CDF of Estimate IOC Date for Elements. 
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Figure 4. Contributors by ATDP to Delay in an Element. 


VI. Conclusion 

The modeling of schedule risks associated with advanced technology development delays is a capability that is 
still being matured but, as evidenced in this paper, already provides insight into the expected behavior of candidate 
and planned technology development investments. Discussion herein pointed out that consideration and inclusion of 
historical data is essential in understanding such risks and that the analysis methodology described by Mathias at al. 
and Forgie and Evans can be taken a step further by enabling analysis of inter-related elements within an 
architecture. The methodology also accounts for the impacts of multiple advanced technology developments for 
each individual architecture element. The current modeling capability leverages a significant amount of historical 
data collected by Mathias et al. and Forgie and Evans, and is in the process of being updated to reflect the actual 
inter-relationships between elements. Future work also includes translating modeled schedule delays into actual 
affordability impacts with emphasis on the fact that delays within one element can often lead to both cost overruns 


11 

American Institute of Aeronautics and Astronautics 


within that element’s development effort as well as other elements in which it shares a schedule relationship and 
related affordability constraints. 

The end product will provide decision makers insight into planned schedules and costs as a function of historical 
data analogies to current technology development challenges. Emphasis will be placed on complete visibility into 
the historical data points that drive the results to improve credibility and insight. Users of the model will also have 
the capability to dynamically choose between historical data to focus on the examples they feel are the most 
analogous to the problem at hand. 
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